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(54) Digital watermark data embedding method and embeddability analyzing method 



(57) To lighten the processing load of embedding a 
digital watermark in content data, a preliminary process 
(embedding strength map generating process) run by a 
preprocessing section and the main process (digital wa- 
termark embedding process) run by a man processing 



section are run mutually independently. Specifically, the 
preliminary process of the preprocessing section is run 
each time new content data is stored to the content data 
database, and the main process of the main processing 
section is run when a content delivery request is re- 
ceived from a receiving terminal (client). 
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Description 

[0001 ] This application claims a priority based on Jap- 
anese Patent Application No.2000-205512 filed on July 
6, 2000, the entire contents of which are incorporated 
herein for all purposes. 

BACKGROUND OF THE INVENTION 

[0002] The present invention relates to a digital wa- 
termarking technology for embedding information in still 
images, moving picture (video) images, and other types 
of content. 

[0003] The need to ensure copyright protection of still 
pictures, video content, and other digitally recorded con- 
tent has driven recent advances in digital watermarking 
technology. Digital watermarking is a technology for em- 
bedding a digital watermark in content data, which is typ- 
ically accomplished by changing a feature value of a 
specific part of the content data according to predefined 
rules for each bit of the digital watermark bitstream. With 
this method the digital watermark cannot be extracted 
from the content data without using at least the above 
rules. For example, if information about the creator (au- 
thor) of the image data is embedded as the digital wa- 
termark information to a location in the image data de- 
termined according to the above rules in a format where- 
by the digital watermark is not visible, the author of the 
image data can be identified by extracting this informa- 
tion from the place defined according to said rules in the 
image data. 

[0004] The principle used by a conventional digital 
watermarking technology to embed information in im- 
age data is described below with reference to Fig. 7. 
[0005] Referring to Fig. 7, an embeddability analysis 
process (further described below) is applied to the orig- 
inal image data 500 to generate an embedding strength 
map 501 . This embedding strength map 501 indicates, 
for each part of the original image data 500, how much 
a selected feature value (such as luminance) can be 
changed to embed the digital watermark in a particular 
part of the original content data (step S5001). 
[0006] The bits of the watermark data 502 bitstream 
are then arranged to the parts in the original image data 
selected according to the predefined rules, and a water- 
mark pattern 503 is generated (step S5002). Note that 
this map uses +1 lo indicate an arranged bit wilh a bil 
value of 1 , -1 to indicate an arranged bit with a bit value 
of 0, and 0 to indicate where no bit is arranged. 
[0007] The embedding strength map 501 and water- 
mark pattern 502 are then multiplied together to produce 
watermark pattern 504. Watermark pattern 504 indi- 
cates the change in the selected feature value at each 
location, and is used to embed each bit of the watermark 
data 502 to a corresponding location in the original im- 
age data (step S5003). 

[0008] The original image data 500 and watermark 
pattern 504 are then merged. This changes the feature 



value of each part (image area) of the original image 
data where the watermark is located with the amount of 
change determined by the watermark pattern 504 data 
for that part (image area) (step S5004). 

5 [0009] The embeddability analyzing process used in 
step S5001 above is described next. As noted above, 
the embeddability analyzing process is a process for 
generating an embedding strength map, which is a table 
of values obtained by measuring, for all parts of the con- 

10 tent data, the allowed change in the feature value of the 
parts that can be used for embedding a watermark. For 
simplicity below let the content data be image data, and 
luminance be the feature value used for embedding bit 
values. 

is [0010] The allowed change in the feature value in this 
example is the amount of change in the luminance (that 
is, a gradation change) that will not be visible to the eye. 
Referring to Fig. 7, experience has demonstrated that a 
large change in luminance is not visibly conspicuous in 
image areas A with a large high frequency component 
(such as dark hair), but is visibly obvious in areas B 
where there are few high frequency components (such 
as the cheeks) and in contour areas C. It is further known 
that while the luminance change is great in all directions 
in image areas where the high frequency component is 
large, there is little change in the luminance value in all 
directions where the high frequency component is small, 
and while the luminance change is great along vectors 
crossing contours, there is little luminance change along 
vectors following the contour. These characteristics can 
therefore be used to detect the luminance change on a 
plurality of predetermined vectors for each part of the 
image data, and set the allowed change in the feature 
value for each said part so that the allowed change be- 
comes smaller as the smallest change in the feature val- 
ue becomes smaller. 

[0011] Embeddability analyzing processes are de- 
scribed more fully in the following paper written by the 
present inventors and others, "Method for maintaining 
digital watermark image quality based on contour pres- 
ervation" (in Japanese), Echizen Isao, Yoshiura Hiroshi, 
Anzai Kohzuke, Taguchi Jun'ichi, Kurozu Yutaka, Sas- 
aki Ryoichi,Tezuka Satoru, Transactions of the Informa- 
tion Processing Society of Japan, Vol. 41 , No. 6, pp 1 828 
- 1 839, 2000. ! 
[0012] Conventionally, the sequence of steps S5001 
lo S5004 shown in Fig. 7 is performed as a single proc- 
ess for embedding digital watermark data in image data. 
That is, all steps of this embeddability analyzing process 
are performed each time a digital watermark is embed- 
ded in the image data. 

[001 3] When delivering or distributing content data to 
plural users, the need to detect illegal copies has in par- 
ticular made it desirable to embed user information (that 
is, information about the purchaser to whom the content 
data is delivered or distributed) in the content data. This 
means that the embeddability analyzing process ac- 
cording to the related art as described above is per- 
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formed for the same content data every time the content 
data Is distributed or delivered. As further noted above, 
this embeddability analyzing process includes a proce- 
dure for measuring the allowed change in a particular 
feature value for each part or area of the content data. 
This means that the processing overhead of the pattern 
generating process, for example, is high. Repeating this 
process the same number of times the content data is 
distributed or delivered simply makes the processing 
load that much greater. 

[0014] There is also the desire to embed watermark 
data in real time to image data captured with a video 
camera for live broadcasting, for example, so that the 
live video is broadcast with the watermark embedded in 
the signal. Copyright protection concerns make it further 
preferable to embed this watermark data in each frame 
of image data output sequentially from the video cam- 
era. However, the high overhead of the above-described 
embeddability analysing process and need for compat- 
ibility with extant systems means that an extremely high 
priced, high performance processing apparatus is need- 
ed by the related art. 

SUMMARY OF THE INVENTION 

[0015] Therefore, the present invention provides 
technology for embedding a digital watermark to content 
data with greater efficiency than the related art. More 
particularly, the present invention relates to technology 
for reducing the processing load of the embeddability 
analyzing process performed in conjunction with em- 
bedding different watermark data to the same content 
data. The load of the embeddability analyzing process 
itself, and particularly the load of the embeddability an- 
alyzing process applied to image data consisting of plu- 
ral data frames, is reduced. 

[0016] A first aspect of the present invention sepa- 
rates the process for embedding watermark data to con- 
tent data into two processes, a preparation process and 
a main process. 

[0017] An embeddability analyzing process is applied 
to the content data in the preliminary process. This proc- 
ess creates a embedding strength map containing val- 
ues indicating the change allowed in a feature value for 
each part of the content data. This allowed change value 
is the change permitted in the feature value to embed 
the bits of Ihe digital walermark dala bil sequence. The 
resulting map is then stored with correlation to the con- 
tent data. The main process embeds the digital waterr 
mark data in the content data by changing, for each bit 
of the digital watermark data bit sequence, the feature 
value of the content part determined according to spe- 
cific rules. The specific change in the feature value is 
determined according to the bit value of the digital wa- 
termark data bit, and the allowed change in the feature 
value stored of that part in the embedding strength map 
stored with correlation to the same content data. 
[0018] The preliminary process is accomplished for 



each content data unit. The main process is applied to 
the content data independently of the preliminary proc- 
ess for each digital watermark. 

[001 9] The embeddability analyzing process is there- 
5 fore needed only once to embed plural different digital 
watermarks to the same content data at different times, 
and the associated processor load is therefore lower 
compared with a method according to the related art. 
The embedding strength map for any same content data 
10 is the same irrespective of the digital watermark data to 
be embedded. Therefore, the embedding strength map 
compiled by the preliminary process for particular con- 
tent data can be used for each of plural different digital 
watermarks to be embedded in the same content data. 
is [0020] An embeddability analyzing process according 
to a second aspect of the invention compiles an embed- 
ding strength map as follows. 

[0021] For at least one of embedding parts of bits of 
the electronic data, the allowed change in a feature val- 
ue of a particular part of the content data corresponding 
to the embedding part is first determined (a first step). 
Said embedding parts are determined by predetermined 
rules (used to generate the watermark pattern). Namely, 
said embedding parts arc inducated by the watermark 
pattern. 

[0022] Based on the obtained allowed change values, 
the allowed change is estimated for each of the embed- 
ding parts other than the at-least-one embedding part 
indicated by the watermark pattern (second step). 
[0023] The simplest estimating method when allowed 
change values are obtained for two or more embedding 
parts, for example, is to use the average of the meas- 
ured values as the allowed change for each of the other 
embedding parts indicated by the watermark pattern. 
The embedding strength map is then compiled using the 
measured or estimated allowed changes for all of the 
embedding parts indicated by the watermark pattern 
(third step). To prevent the watermark pattern from being 
derived from the embedding strength map, a desired al- 
lowed change is preferably also set for parts other than 
the embedding parts indicated by the watermark pat- 
tern. 

[0024] Compared with measuring the allowed change 
values for all parts of the content data as in a conven- 
tional embeddability analyzing process as described 
above, the method of the present invention enables fast- 
er processing because the location for which the al- 
lowed feature value change values are actually meas- 
ured is at least one of the embedding parts indicated by 
the watermark pattern. 

[0025] An embeddability analyzing process for creat- 
ing an embedding strength map for content data com- 
prising plural frames according to a third aspect of the 
present invention is described next. 
[0026] A plural frame satisfying specific conditions is 
sequentially selected from the beginning of the plural 
data frames constituting the content data (first step). For 
example, one data frame is captured every specific 
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number of frames in sequence from the beginning of the 
frames in the content data. Alternatively, the similarity 
between a candidate data frame and a reference data 
frame is detected, and if the similarity is less than a par- 
ticular threshold value, the candidate data frame is cap- 
tured while resetting the candidate data frame as the 
reference frame for comparison with the nextcaptured 
candidate frame. An embedding strength map is then 
generated for the captured frame. As described above, 
this embedding strength map stores the allowed change 
in the feature value of the content data when embedding 
the bits of the digital watermark data in the captured 
frame (second step). This embedding strength map is 
then used as the embedding strength map for all frames 
from the captured frame for which the map was com- 
piled to the frame before the next captured frame for 
which a different map is compiled (third step). 
[0027] Thus comprised, it is not necessary to gener- 
ate a embedding strength map for every frame of con- 
tent data comprising plural data frames. Processing is 
therefore faster than the embeddability analyzing proc- 
ess according to the related art. 

[0028] These and other benefits are described 
throughout the present specification. A further under- 
standing of the nature and advantages of the invention 
may be realized by reference to the remaining portions 
of the specification and the attached drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0029] 

Fig. 1 is a schematic diagram of a content distribu- 
tion system in which the first embodiment of the 
present invention is used; 

Fig. 2 is' a schematic diagram of the content distri- 
bution center 1 shown in Fig. 1 ; 
Fig. 3 is a flow chart of the operation of the content 
distribution center 1 shown in Fig. 2; 
Fig. 4 is a block diagram of a typical hardware con- 
figuration of the content distribution center 1 shown 
in Fig. 2; 

Fig. 5 is a schematic diagram of a video camera in 
which the second embodiment of the present inven- 
tion is used; 

Fig. 6 is a flow chart of the process for storing an 
embedding strength map lo the embedding strength 
map memory 63 of the video camera shown in Fig. 
5; and 

Fig. 7 is used to describe the principle of embedding 
data to image data using a digital watermarking 
technology according to the related art. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0030] A first preferred embodiment of the present in- 
vention is described below with reference to the accom- 



panying figures using by way of example the digital wa- 
termark embedding method of the present invention as 
applied in a content distribution system. 
[0031] Fig. 1 is a schematic diagram of a content dis- 

5 tribution system in which the first embodiment of the 
present invention is used. As shown in Fig. 1 , this con- 
tent distribution system comprises a content distribution 
center 1 connected to a plurality of receiving terminals 
2 by way of an intervening network 3. 

10 [0032] The content distribution center 1 applies an 
embeddability analyzing process to all content available 
for distribution to generate corresponding embedding 
strength maps, and stores the embedding strength 
maps correlated to the particular content. When a con- 

15 tent delivery request is then received from a receiving 
terminal 2, the content distribution center 1 creates a 
digital watermark containing user information, that is, in- 
formation about the receiving terminal 2 that sent the 
. content delivery request.. It then embeds this digital wa- 

20 termark in the content requested for delivery according 
to the embedding strength map prepared for the re- 
quested content and predefined rules (rules for gener- 
ating a watermark pattern), and thus generates water- 
marked content. This watermarked content is then 

25 transmitted to the receiving terminal 2 that sent the con- 
tent delivery request. 

[0033] This content distribution center 1 is described 
in further detail below. The receiving terminal 2 can be 
any device that has a function for communicating with 
30 the content distribution center 1 via the network 3, and 
detailed description thereof is thus omitted below. 
[0034] Fig. 2 is a block diagram of the content distri- 
bution center 1 . 

[0035] As shown in Fig. 2 the content distribution cent- 
os er 1 has an input/output section 11 , preprocessing sec- 
tion 13, main processing section 14, storage unit 15, and 
a controller 12 for systematically controlling these vari- 
ous components. The I/O section 11 accepts original 
content data from a keyboard or other input device or a 
40 network 3 for delivery to a receiving terminal 2, and 
transmits watermarked content to the receiving terminal 
2 via the network 3. 

[0036] The storage unit 15 has a content database 
151 for storing original content data; an embedding 

^5 strength map database 152 for storing the embedding 
strength maps for each content stored to content data- 
base 151; and watermarking rules storage 153 for stor- 
ing the watermarking rules used for generating the wa- 
termark pattern embedded to the content data. 

so [0037] The preprocessing section 13 has an embed- 
dability analysis processing section 131 for applying an 
embeddability analyzing process to each content data 
stored in content database 151; and an embedding 
strength map generator 132 for generating the embed- 

55 ding strength maps based on the results returned by the 
embeddability analysis processing section 131 . 
[0038] The main processing section 14 has a water- 
mark data generator 1 41 for generatin g watermark data , 
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a watermark pattern generator 142, and a watermarked 
content generator 143. The watermark pattern genera- 
tor 142 creates a watermark pattern by converting the 
watermark data produced by the watermark data gen- 
erator 141 to a watermark pattern according to the wa- 
termarking rules stored to watermarking rules storage 
1 53. The watermarked content generator 1 43 produces 
the watermarked content by embedding the watermark 
data in the content data using the embedding strength 
map stored in the embedding strength map database 
1 52 and the watermark pattern generated by the water- 
mark pattern generator 142. 

[0039] Fig. 3 is a flow chart of the operation of the con- 
tent distribution center 1 as described above. As will be 
known from Fig. 3, the content distribution center 1 ac- 
complishes two processes: a preliminary process (a) ex- 
ecuted by the preprocessing section 13, and a main 
process (b) accomplished by the main processing sec- 
lion 14. 

[0040] The preliminary process (a) is described first 
below. 

[0041 ] When new content data (original content data) 
is stored from the I/O section 11 to the content database 
151 (step S1001), the controller 12 instructs the pre- 
processing section 13 to create an embedding strength 
map for the newly stored content data. 
[0042] When the preprocessing section 13 receives 
this command, embeddability analysis processing sec- 
tion 131 reads the newly stored content data from con- 
tent database 151, and reads the watermarking rules 
from watermarking rules storage unit 1 53 (step S1 002). 
[0043] The content data is divided into unit blocks, 
which are the unit areas to which one bit of the water- 
mark data (a bit sequence) is embedded. For at least 
one of unit blocks (unit blocks to which the bits of the 
watermark data are embedded) defined according to the 
watermarking rules, the allowed change in the feature 
value (such as luminance value or chrominance value 
if the content is image data) of a unit block is then ob- 
tained (step S1003). Note that this allowed change ob- 
taining operation is basically the same as that of the em- 
beddability analyzing process described in the related 
art. 

[0044] Next, for each unit block determined according 
to the watermarking rules other than the unit blocks for 
which the allowed change was measured in step S1 003, 
the embeddability analysis processing section 131 es- 
timates the allowed change the feature value of the unit 
block based on the allowed change data obtained in 
step S1003 (step S1004). For example, if the allowed 
change is measured for only one block in step S1003, 
the value of that change is used as the estimate value. 
If the allowed change is measured for two or more 
blocks in step S1 003, the average of the measured val- 
ues is used as the estimate value. 
[0045] Note, further, that if the allowed change in the 
feature value is measured for each of the unit blocks 
defined according to the watermarking rules in step 



S1003, the estimation step in step S1004 is unneces- 
sary. 

[0046] The embedding strength map generator 132 
writes the allowed change in the feature value of each 
5 unit block obtained in step S1003 or estimated in step 
S1004 to the corresponding unit blocks determined ac- 
cording to the watermarking rules in a map obtained by 
dividing the content data into the unit blocks as noted 
above. The result of this step is an embedding strength 

10 map 501 as shown in Fig. 7 (step S1005). So that the 
watermarking rules cannot be deduced from the embed- 
ding strength map 501 , a particular allowed change val- 
ue is preferably set for each of the unit blocks other than 
the unit blocks determined according to the watermark- 

15 ing rules in the embedding strength map 501 . 

[0047] After generating the embedding strength map 
501 as described above, the embedding strength map 
generator 1 32 stores the map in the embedding strength 
map database 152 so thai the correlation to Ihe content 

20 data read in step S1 002 can be known (step S1006). 
[0048] The main process (b) is described next. 
[0049] Note that this main process (b) is run inde- 
pendently of the above preliminary process (a). 
[0050] When a content delivery request is received 

25 from the receiving terminal 2 by way of I/O section 11 
(step S2001 ), the controller 1 2 passes the user informa- 
tion and content identification data contained in the re- 
quest to the main processing section 14, and instructs 
the main processing section 14 to generate water- 

30 marked content. 

[0051] The watermark data generator 141 then gen- 
erates watermark data containing the user information 
and content identification data received from the con- 
troller 12 (step S2002). and passes the watermark data 

35 to he watermark pattern generator 142. The watermark 
pattern generator 142 then reads the watermarking 
rules from watermarking rules storage 153. The water- 
mark pattern generator 1 42 then arranges the bits of the 
bit sequence representing the watermark data to the unit 

40 blocks of the content (the unit blocks to which the bits 
of the watermark data are to be embedded) determined 
according to the watermarking rules, and generates a 
watermark pattern 503 as shown in Fig. 7 (step S2003). 
As described above with reference to Fig. 7, +1 indi- 

45 cates a arranged bit with a bit value of 1 , -1 indicates an 
arranged bit with a bit value of 0 bit, and 0 indicates 
where no watermark data bil is arranged. 
[0052] After the watermark pattern 503 is produced, 
watermarked content generator 143 reads the embed- 

50 ding strength map 501 for the content identified by the 
content identification data passed from the controller 12 
from the embedding strength map database 152, and 
multiplies the embedding strength map 501 with the wa- 
termark pattern 503. This results in a watermark pattern 

55 504 as shown in Fig. 7 (step S2004). Note that this wa- 
termark pattern 504shows the change in the feature val- 
ue of each unit block whereby each bit of the watermark 
data is embedded in the corresponding unit blocks of 
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the requested content. The content data identified by the 
content identification data passed from the controller 12 
is then read from content database 1 51 and merged with 
the watermark pattern 504 to change the feature value 
of each unit block of the content data the amount indi- 
cated for that unit block by the watermark pattern 504. 
The result is watermarked content having the watermark 
data generated by the watermark data generator 141 
embedded therein (step S2005). 
[0053] The watermarked content is then transmitted 
through the I/O section 11 to the receiving terminal 2 
that sent the content delivery request (step S2006). 
[0054] The content distribution center 1 can be 
achieved in hardware using a combination of memory 
and an integrated logic circuit such as an ASIC or FPGA 
device. The content distribution center 1 can also be 
achieved as a software program that runs on a common 
computer system having a CPU 801 , memory 802, ex- 
ternal storage device 803 such as a hard disk, a reader 
804 for reading data from a CD-ROM or other data stor- 
age medium 810, a keyboard or other input device 805, 
a display or other output device 806, a communications 
device 807 for communicating with another device via 
a network 3, and an interface 808 for controlling data 
communication between these various components. Al- 
so, a program for achieving the content distribution cent- 
er 1 in software is installed to the external storage device 
803 from data storage medium 810 by the reader 804, 
or from the network 3 by the communications device 
807, and is then loaded into memory 802 and run by the 
CPU 801. 

[0055] A first preferred embodiment of the present in- 
vention is described above. 

[0056] As will be known from the preceding descrip- 
tion, a preliminary process, that is, an embedding 
strength map generating process, run by the preproc- 
essing section 13, and the main process, that is, the wa- 
termark data embedding process, run by the main 
processing section 14, are independent processes in 
this exemplary embodiment of the present invention. 
More specifically, the preliminary process run by the pre- 
processing section 1 3 is performed every time new con- 
tent data is stored to the content database 151 , and the 
main process run by the main processing section 14 is 
performed whenever a content delivery request is re- 
ceived from a receiving terminal 2. The preliminary proc- 
ess is therefore performed only once for the same con- 
tent data, even when plural receiving terminals 2 re- 
quest delivery of the same content data. The present 
invention can therefore efficiently generate water- 
marked content compared with the watermarked con- 
tent generating process according to the related art as 
described above, 

[0057] Furthermore, the embeddability analyzing 
process for creating the embedding strength map is ap- 
plied to at least one of the unit blocks determined ac- 
cording to the watermarking rules, and for each of the 
remaining unit blocks the allowed change is estimated 



from the results of the embeddability analyzing process 
applied to at least one of said unit blocks. Therefore, the 
unit blocks for which the allowed feature value chang 
is actually measured are the unit blocks to which bits of 
5 the watermark data are actually embedded. In other 
words, unnecessary measurement of the allowed fea- 
ture value change can be omitted for unit blocks to which 
the bits of the watermark data are not actually embed- 
ded. This process can therefore be accomplished faster 
10 than the embedding strength map generating process 
described according to the related art. 
[0058] The present embodiment is described using by 
way of example a case in which the watermarked con- 
tent generating step of the main processing section 14 

15 js accomplished each time a content delivery request is 
received from a receiving terminal 2, but the invention 
shall not be so limited. When the same content is dis- 
tributed to a plurality of receiving terminals 2, the present 
invention can be used to change the watermark data for 

20 each receiving terminal 2. 

[0059] While the preprocessing section 13 and main 
processing section 14 are disposed to the same appa- 
ratus in the above preferred embodiment, these can be 
allocated to separate apparatuses. For example, in the 

25 Fig, 1 f the content distribution center 1 employs a con- 
figuration in which the main processing section 14 is re- 
moved in Fig. 2. In this case the requested content (orig- 
inal content data) and corresponding embedding 
strength map are read from the storage unit 15 in re- 

30 sponse to a delivery request from a receiving terminal 
2, and transmitted to said receiving terminal 2. On the 
other hand, in Fig. 1 , the receiving terminal 2 employs 
a configuration in which the preprocessing section 13 is 
removed in Fig. 2, and stores to the storage unit 15 the 

35 original content data and corresponding embedding 
strength map transmitted from the content distribution 
center 1 in response to a delivery request from the re- 
ceiving terminal 2. The main processing section 14 is 
then activated to generate the watermarked content, 

40 which is passed through I/O section 11 and delivered to 
the user at the receiving terminal 2. If the receiving ter- 
minal 2 is an on-line terminal installed in a convenience 
store, for example, an efficient delivery system can be 
constructed with the processes relating to embedding 

45 watermark data (that is, the embedding strength map 
generating process and the watermark data embedding 
process) distributed between the content distribution 
center 1 and receiving terminal 2. 
[0060] A digital watermark embedding method ac- 

50 cording to a second preferred embodiment of the 
present invention is described next below using appli- 
cation in a digital video camera by way of example. 
[0061] Fig. 5 is a schematic diagram of a video cam- 
era in which this second embodiment of the present in- 

55 vention is applied. 

[0062] Referring to Fig. 5, a video signal representing 
the image output from an imaging unit 51 is digitized by 
A/D converter 52 and converted to digital image data 
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consisting of a sequence of frame data. The frame data 
is sequentially output from the A/D converter 52 to buffer 
53. The buffer 53 has capacity sufficient to hold frame 
data for at least one image frame, and adjusts the frame 
data input timing to the watermark embedding section 
54. 

[0063] The watermark embedding section 54 moni- 
tors the embedding strength map memory 63. When the 
content of the embedding strength map memory 63 is 
updated, the watermark embedding section 54 multi- 
plies the watermark pattern 503 stored to watermark 
pattern memory 58 with the updated embedding 
strength map 501 stored to the embedding strength map 
memory 63 to generate and store watermark pattern 
504 shown in Fig. 7. As noted above, the watermark pat- 
tern 504 indicates the change in the particular feature 
value of each unit block in the content and is used to 
embed each bit of the watermark data 502 to a corre- 
sponding unit block in the content data. The watermark 
pattern 504 and frame data output from the buffer 53 are 
then merged, thereby changing the feature value of 
each unit block (the unit blocks to which the watermark 
data bits are to be embedded) in the frame data the 
amount of change indicated by the watermark pattern 
504 for that unit block. The result is watermarked frame 
data in which the watermark data is embedded. 
[0064] The encoder 55 then encodes and outputs the 
watermarked frame data from watermark embedding 
section 54 according to the MPEG standard or other pro- 
tocol. 

[0065] In a video camera thus comprised storing the 
watermark pattern to the watermark pattern memory 58 
is accomplished as follows before the watermarked 
frame data is generated. 

[0066] The watermark data generator 56 generates 
the watermark data and outputs it to the watermark pat- 
tern generator 57. The watermark data in this case in- 
cludes information about the videographer. for example, 
input using a control panel or other means not shown in 
the figures. The watermark pattern generator 57 reads 
the watermarking rules from the watermarking rule 
memory 64. As noted above, the watermarking rules 
control generating the watermark pattern to be embed- 
ded in the frame data. 

[0067] The watermark pattern generator 57 then ar- 
ranges the bits of the bit sequence representing the wa- 
termark data received from the watermark data gener- 
ator 56 to unit blocks of frame data (trie unit blocks to 
which the bits of the watermark data are to be embed- 
ded) determined according to the watermarking rules. 
This results in a watermark pattern 503 such as shown 
in Fig. 7. As described above with reference to Fig. 7, 
+1 indicates an arranged bit with a bit value of 1 , -1 in- 
dicates an arranged bit with a bit value of 0, and 0 indi- 
cates where no watermark data bit is arranged. The re- 
sulting watermark pattern 503 is then stored to water- 
mark pattern memory 58. 

[0068] Storing the embedding strength map to the 



embedding strength map memory 63 in the above video 
camera is described next below with reference to the 
flow chart in Fig. 6. 

[0069] Similarity detector 60 monitors the buffer 53. 
5 When new frame data (the "current frame data" below) 
is stored to buffer 53 (step S3001), similarity detector 
60 detects the similarity between the current frame data 
and the frame data stored to the reference image mem- 
ory 59 (referred to as the "reference frame data" below) 
to determine if the current frame data can be treated as 
frame data with the same content as the reference frame 
data (that is, no motion is detected between the image 
frames) (step S3002). This can be accomplished using 
a motion prediction process such as that used in MPEG 
encoding to predict motion in each macroblock of the 
current frame data relative to the reference frame data. 
The sum of motion in each macroblock of the current 
frame data with respect to the corresponding macrob- 
lock in the reference frame data is then calculated, and 
the resulting sum is compared with a specific threshold 
value. If the sum of motion is less than this threshold 
value, the current frame data is determined to contain 
the same content as the reference frame data (that is, 
there is no motion therebetween), and is thus frame data 
with a strong correlation (similarity) to the reference 
frame data. The current frame data is otherwise deter- 
mined to not contain the same content (that is, there is 
motion between corresponding frames), and is thus 
frame data with a weak correlation (similarity) to the ref- 
erence frame data. 

[0070] If step S3002 determines that the current 
frame data is frame data with a strong correlation to the 
reference frame data (step S3003 returns yes), the sim- 
ilarity detector 60 orders output of the current frame data 
stored to the buffer 53 to the watermark embedding sec- 
tion 54 (step S3004). Control then loops back to step 
S3001 and waits for new current frame data to be stored 
to the buffer 53. 

[0071] However, if step S3002 determines that the 
current frame data is frame data with a weak correlation 
to the reference frame data (step S3003 returns no), the 
similarity detector 60 instructs the embeddability analy- 
sis section 61 to create an embedding strength map. 
[0072] The embeddability analysis section 61 then 
reads the watermarking rules from the watermarking 
rule memory 64. The current frame data is divided into 
unit blocks, which are Ihe unit areas to which one bit of 
the watermark data (a bit sequence) is embedded. The 
allowed change in the feature value (such as luminance 
value or chrominance value if the content is image data) 
of a unit block is then obtained, according to the same 
manner as the embeddability analysis process de- 
scribed above for al least one of unit blocks (unit blocks 
to which the bits of the watermark data are embedded) 
defined according to the watermarking rules (step 
S3005). 

[0073] Next, for each of the unit blocks determined ac- 
cording to the watermarking rules other than the unit 
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blocks for which the allowed change was obtained in 
step S3005, the embeddability analysis section 61 esti- 
mates the allowed change in the feature value of the unit 
block based on the allowed change values obtained in 
step S3005 (step S3006). This is similar to step S1004 
in Fig. 3. 

[0074] The embedding strength map generator 62 
writes the allowed change in the feature value of each 
unit block obtained in step S3005 or estimated in step 
S3006 to the corresponding unit blocks determined ac- 
cording to the watermarking rules in a map obtained by 
dividing the content data into the unit blocks as noted 
above. This is similar to step S1 005 in Fig. 3. The result 
of this step is an embedding strength map 501 as shown 
in Fig. 7 (step S3007). 

[0075] Once an embedding strength map 501 is thus 
compiled, the embedding strength map generator 62 
updates the embedding strength map stored in the em- 
bedding strength map memory 63 to the new map (step 
S3008). 

[0076] Once the updated embedding strength map is 
stored to the embedding strength map memory 63, the 
similarity detector 60 updates the reference frame data 
stored to the reference image memory 59 to the current 
frame data (step S3009). The buffer 53 is controlled to 
output the current frame data to the watermark embed- 
ding section 54 (step S3004). Control then loops back 
to step S3001 , and the process waits until new current 
frame data is stored to the buffer 53. 
[0077] It should be noted that, like the content distri- 
bution center 1 described as a first embodiment of the 
present invention, the digital video camera of this em- 
bodiment can be achieved in hardware using a combi- 
nation of memory and an integrated logic circuit such as 
an ASIC or FPGA device. It can also be achieved as a 
software program that runs on a common computer sys- 
tem having a CPU 801 , memory 802, external storage 
device 803, a data reader 804, an input device 805 such 
as a video player for outputting a video signal containing 
captured images, a display or other output device 806, 
and an interface 808 for controlling data communication 
between these various components as shown in Fig. 4. 
[0078] A second preferred embodiment of the present 
invention is described above. 

[0079<. In this exemplary embodiment of the present 
invention, current frame data sequentially output from 
an imaging unit 51 is used as content to be water- 
marked. If the current frame data can be handled as 
frame data of the same content as the reference frame 
data, that is, there is no motion between the frames, the 
embedding strength map compiled for the reference 
frame data is used as the embedding strength map for 
embedding watermark data to the current frame data. It 
is therefore not necessary to actually generate an em- 
bedding strength map for each of the plural frame data 
constituting the content data. Processing is therefore 
faster compared with a conventional embeddability an- 
alyzing process. 



[0080] This embodiment of a digital watermark em- 
bedding method according to the present invention is 
described with application in a digital video camera, but 
the invention shall obviously not be so limited. For ex- 

5 ample, the digital watermark embedding method of the 
invention can be used in a wide range of electronic de- 
vices, including digital video recorders. 
[0081] When the digital watermark embedding meth- 
od of the present invention is used in a digital video re- 

10 corder, an interface for capturing digital video signals is 
provided in place of the imaging unit 51 and A/D con- 
verter 52 in Fig. 5, and a recording section for recording 
a digital video signal comprising a plurality of water- 
marked frames is disposed in place of the encoder 55. 

15 [0082] This embodiment of our invention as described 
above detects the similarity between reference frame 
data and the current frame data sequentially output from 
the imaging unit 51 as part of the content.data, and de- 
termines whether to use the embedding strength map 

20 for the reference frame data as the embedding strength 
map for embedding watermark data to the current frame 
data. The invention shall not, however, be so limited. For 
example, an embedding strength map can be generated 
for every n-th frame occurring in the sequential output 

25 stream of frame data from the imaging unit 51 and used 
for embedding watermark data to each subsequent 
frame until the next embedding strength map is gener- 
ated. 

[0083] In this case the reference image memory 59 

30 and similarity detector 60 shown in Fig. 5 are replaced 
by a counter. The counter is incremented one each time 
new current frame data is stored to the buffer 53. When 
the count reaches a specific value, the embeddability 
analysis section 61 is instructed to generate an embed- 

35 ding strength map and the counter is then reset. 

[0084] The present invention shall not be limited to the 
above described embodiments, and the invention can 
be changed in various ways without departing from the 
scope of the following claims. 

40 [0085] For example, the process performed by the 
preprocessing section 1 3 in the first embodiment for cre- 
ating an embedding strength map can apply an embed- 
dability analyzing process to each part of the content 
data as described in the related art to obtain the allowed 

45 feature value change for each said part. When the con- 
tent data contains plural frames, such as moving picture 
data (video), a method as described in the second em- 
bodiment can alternatively be used. 
[0086] The process shown in Fig. 6 is run before de- 

50 livering the content data to compile an embedding 
strength map suitable for each frame of the content data . 
These maps arc then stored to the embedding strength 
map database 1 52 with correlation to the frame data to 
which each embedding strength map applies. 

55 [0087] While these exemplary embodiments of the in- 
vention have been described using image data as the 
content data, the present invention can be applied to 
various other types of content data, including audio data 
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and text data. 

[0088] As described above, the present invention re- 
duces the processing load of the embeddability analyz- 
ing process required to embed different watermark data 
to the same content data. In addition, the processing 
load of the embeddability analyzing process itself is also 
reduced. 

[0089] The specification and drawings are, according- 
ly, to be regarded in an illustrative rather than a restric- 
tive sense. It will, however, be evident that various mod- 
ifications and changes may be made thereto without de- 
parting from the broader spirit and scope of the invention 
as set forth in the claims. 



Claims 

1 . A digital watermark embedding method for embed- 
ding digital watermark data to content data, com- 
prising: 

a preliminary process step for applying an em- 
beddability analyzing process to content data 
before embedding digital watermark data to the 
content data, compiling an embedding strength 
map for each part of the content data, and stor- 
ing said embedding strength map with correla- 
tion to the content data, 

said embedding strength map indicating, for 
said part of the content data, allowed change 
in a feature value of said part, said allowed 
change being the change in the feature value 
allowed for embedding a bit of a bit sequence 
representing the digital watermark data in said 
part; and 

a main process for embedding the digital wa- 
termark data in the content data by, for each bit 
of the bit sequence representing the digital wa- 
termark data, changing the feature value of a 
part in the content data determined according 
to predetermined rules an amount determined 
according to the bit value of said bit and the al- 
lowed change in the feature value of said part 
indicated by the embedding strength map 
stored for the particular content data; 

wherein the main process is accomplished 
separately from the preliminary process for each of 
plural different digital watermark data. 

2. A digital watermark embeddability analyzing meth- 
od for generating an embedding strength map indi- 
cating for each part of content data allowed change 
in a feature value of said part, said allowed change 
being the change in the feature value allowed for 
embedding a bit of a bit sequence representing the 
digital watermark data in said part of the content da- 
ta, the embeddability analyzing method comprising: 



a first step for obtaining, for at least one em- 
bedding part determined according to predeter- 
mined rules for embedding bits of the digital wa- 
termark data, an allowed change in a feature 
5 value of said part of the content data; 

a second step for estimating an allowed change 
in a feature value of each content data part oth- 
er than said at least one content part for em- 
bedding bits of the digital watermark data 
based on the allowed feature value change ob- 
tained in the first step for at least one content 
part for embedding the bits of the digital water- 
mark data; and 

a third step for generating an embedding 
strength map using the allowed change data 
obtained in the first step for at least one content 
part for embedding the bits of the digital water- 
mark data, and the allowed change estimated 
by the second step for the content parts other 
than the at least one content part for embedding 
the bits of the digital watermark data. 

3. A digital watermark embeddability analyzing meth- 
od for generating an embedding strength map indi- 
cating for each part of content data consisting of plu- 
ral frames an allowed change in a feature value of 
said part, said allowed change being change in the 
feature value allowed for embedding a bit of a bit 
sequence representing the digital watermark data 
in said part of the content data, the embeddability 
analyzing method comprising: 

a first step for capturing one unit of frame data 
satisfying specific conditions in sequence from 
a beginning of a stream of plural frames consti- 
tuting the content data; 

a second step for generating an embedding 
strength map for each part of the frame data 
captured in the first step, the embedding 
strength map indicating change allowed in a 
feature value of said part when embedding a bit 
of the bit sequence representing the digital wa- 
termark data in said part; and 
a third step for allocating the embedding 
strength map compiled in the second step as 
the embedding strength map used for all frame 
data from the frame data used for generating 
the embedding strength map to the frame data 
one frame before the new frame data next cap- 
tured in the first step. 

4. A digital watermark embeddability analyzing meth- 
od as described in claim 3, wherein the first step 
captures one frame data unit each specific number 
of frames in sequence from a beginning of the plural 
frame data units constituting the content data. 

5. A digital watermark embeddability analyzing meth- 
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od as described in claim 3, wherein the first step 
detects a degree of similarity between candidate 
frame data and. reference frame data, and if said 
degree of similarity is less than a specific value, 
captures the candidate frame data and sets said 5 
frame data as the reference frame data for the next 
candidate frame data. 

6. A digital watermark embeddability analyzing meth- 
od as described in claim 3, wherein the second step 10 
uses a digital watermark embeddability analyzing 
method as described in claim 2 to generate the em- 
bedding strength map for frame data captured in the 
first step. 

15 

7. A digital watermark embedding apparatus for em- 
bedding digital watermark data to content data, the 
digital watermark embedding apparatus having a 
preprocessing section and. a main processing sec- 
tion: 20 

the preprocessing section comprising 

a map generator which applies an embed- 
dability analyzing process to content data 25 
and compiling an embedding strength map 
for each part of the content data, said em- 
bedding strength map indicating, for said 
each part of the content data, an allowed 
change in a feature value of said part, said 30 
allowed change being the change in the 
feature value allowed when embedding a 
bit of a bit sequence representing the dig- 
ital watermark data in said part, 
a content database which stores content 35 
for which the map generator compiles em- 
bedding strength maps, and 
a map database which stores the maps 
compiled by the map generator; and 

40 

the main processing section comprising: 

an input unit which inputs content for deliv- 
ery to a content user and an embedding 
strength map for said content from said 45 
content database and said map database, 
a watermark data generator which gener- 
ates watermark data to be embedded in 
content obtained by the input unit, said wa- 
termark data generated in the same so 
number as content users to whom content 
is delivered, and 

a watermarked content generator which 
generates watermarked content in the 
same number as content users to whom 55 
content is delivered by applying a content 
watermarking process using each digital 
watermark generated by the watermark 



data generator, 

said content watermarking process 
generating watermarked content in 
which a digital watermark is embedded 
by changing, for each bit of a bit se- 
quence representing the digital water- 
mark, a feature value of a content part 
determined according to predefined 
rules an amount determined according 
to the bit value of said bit and the al- 
lowed change in the feature value of 
said part as indicated by the embed- 
ding strength map input by the input 
unit. 

8. A digital watermark embeddability analyzing appa- 
ratus for generating an embedding strength map in- 
dicating for each part ol content data an allowed 
change in a feature value of said part, said allowed 
change being the change in the feature value al- 
lowed for embedding a bit of a bit sequence repre- 
senting the digital watermark data in said part of the 
content data, the embeddability analyzing appara- 
tus comprising: 

a first unit which obtains, for at least one em- 
bedding part determined according to predeter- 
mined rules for embedding bits of the digital wa- 
termark data, an allowed change in a feature 
value of said part of the content data; 
a second unit which estimates an allowed 
change in a feature value of each content data 
part other than said at least one content part for 
embedding bits of the digital watermark data 
based on the allowed feature value change ob- 
tained by the first unit for at least one content 
part for embedding the bits of the digital water- 
mark data; and 

a third unit which generates an embedding 
strength map using the allowed change data 
obtained by the first unit for at least one content 
part for embedding the bits of the digital water- 
mark data, and the allowed change estimated 
by the second unit for the content parts other 
than the at least one content part for embedding 
the bits of the digital watermark data. 

9. A digital watermark embeddability analyzing appa- 
ratus for generating an embedding strength map in- 
dicating for each part of content data consisting of 
plural frames an allowed change in a feature value 
of said part, said allowed change being change in 
the feature value allowed for embedding a bit of a 
bit sequence representing the digital watermark da- 
ta in said part of the content data, the embeddability 
analyzing apparatus comprising: 
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a first unit which captures one unit of frame data 
satisfying specific conditions in sequence from 
a beginning of a stream of plural frames consti- 
tuting the content data; 

a second unit which generates an embedding 5 
strength map for each part of the frame data 
captured by the first unit, the embedding 
strength map indicating change allowed in a 
feature value of said part when embedding a bit 
of the bit sequence representing the digital wa- 10 
termark data in said part; and 
a third unit which allocates the embedding 
strength map compiled by the second unit as 
the embedding strength map used for all frame 
data from the frame data used for generating '5 
the embedding strength map to the frame data 
one frame before the new frame data next cap- 
tured by the first unit. 

10. A data storage medium recording a program for em- 20 
bedding digital watermark data in content data, said 
program read into and run on a computer to con- 
struct therein: 

a preprocessing section which applies an em- 25 
beddability analyzing process to content data 
before embedding digital watermark data to the 
content data, compiling an embedding strength 
map for each part of the content data, and 
stores said embedding strength map with cor- 30 
relation to the content data, 
said embedding strength map indicating, for 
each part of the content data, allowed change 
in a feature value of said part, said allowed 
change being the change in the feature value 35 
allowed for embedding a bit of a bit sequence 
representing the digital watermark data in said 
part of the content data; and 
a main processing section which embeds the 
digital watermark data in the content data by, 40 
for each bit of the bit sequence representing the 
digital watermark data, changing a feature val- 
ue of a part in the content data determined ac- 
cording to predetermined rules an amount de- 
termined according to the bit value of said bit 45 
and the allowed change in the feature value of 
said part indicated by the embedding strength 
map stored for the particular content data; 

wherein the main processing section is ac- so 
complished separately from the preprocessing sec- 
tion for each of plural different digital watermark da- 
ta. 

11. A data storage medium recording a program for 55 
generating an embedding strength map indicating 

for each part of content data allowed change in a 
feature value of said part, said allowed change be- 



ing the change in the feature value allowed for em- 
bedding a bit of a bit sequence representing the dig- 
ital watermark data in said part of the content data, 
said program read into and run on a computer to 
construct therein: 

a first unit which obtains, for at least one em- 
bedding part determined according to predeter- 
mined rules for embedding bits of the digital wa- 
termark data, an allowed change in a feature 
value of said part; 

a second unit which estimates an allowed 
change in a feature value of each content data 
part other than said at least one content part for 
embedding bits of the digital watermark data 
based on the allowed feature value change ob- 
tained by the first unit for at least one content 
part for embedding the bits of the digital water- 
mark data; and 

a third unit which generates an embedding 
strength map using the allowed change data 
obtained by the first unit for at least one content 
part for embedding the bits of the digital water- 
mark data, and the allowed change estimated 
by the second unit for the content parts other 
than the at least one content part for embedding 
the bits of the digital watermark data. 

12. A data storage medium recording a program for 
generating an embedding strength map indicating 
for each part of content data consisting of plural 
frames an allowed change in a feature value of said 
part, said allowed change being change in the fea- 
ture value allowed for embedding a bit of a bit se- 
quence representing the digital watermark data in 
said part of the content data : said program read into 
and run on a computer to construct therein: 

a first unit which captures one unit of frame data 
satisfying specific conditions in sequence from 
a beginning of a stream of plural frames consti- 
tuting the content data; 

a second unit which generates an embedding 
strength map for each part of the frame data 
; captured by the first unit, the embedding 
strength map indicating change allowed in a 
feature value of said pari when embedding a bit 
of the bit sequence representing the digital wa- 
termark data in said part; and 
a third unit which allocates the embedding 
strength map compiled by the second unit as 
the embedding strength map used for all frame 
data from the frame data used for generating 
the embedding strength map to the frame data 
one frame before the new frame data next cap- 
tured by the first unit. 
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